Papers with stacked network architectures
Enabling Lightweight Fine-tuning for Pre-trained Language Model Compression based on Matrix Product Operators (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for pre-trained language models (PLMs) use parameter reduction techniques. |
| Approach: | They propose a pre-trained language model compression approach based on the matrix product operator from quantum many-body physics. |
| Outcome: | The proposed approach can decompose an original matrix into central tensors and auxiliary tenses . it can be applied to the original or compressed PLMs in a general way, with a lighter network . |